Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Image recognition, external URLs, content references #44

Merged
merged 30 commits into from
Jan 12, 2024
Merged

Image recognition, external URLs, content references #44

merged 30 commits into from
Jan 12, 2024

Conversation

stoerr
Copy link
Member

@stoerr stoerr commented Jan 12, 2024 •
edited
Loading

This introduces several features for the content creation dialog.

  • If the component itself or siblings contain references to other pages / content fragments / experience fragments / images, up to 5 of these are offered as sources in the source selection menu. If selected, their text content is retrieved into the source text. If an image is referenced, it is displayed instead as source content.
  • There is an additional selector "external URL" that makes an URL field appear. The text content of the URL will be used as source field.
  • If an image is selected from the source menu, it is possible to use the (beta) vision functionality of ChatGPT to e.g. generate a textual description.
  • For the vision functionality an optional configuration for the used OpenAI model is needed (currently gpt-4-vision-preview, still in beta.)

@stoerr
add method to collect links
c58b881
@stoerr
return link paths as content selectors
b4768bd
@stoerr
add case for content references in composum create dialog
fe9b1aa
@stoerr
add exception for resources somehow not handled by HTML renderer into…
c5beace 
… markdown handling
@stoerr
output image references in markdown
f75e0ce
@stoerr
set Imagepath header if source is an image, and display the image
c97561a
@stoerr
image encoding done. But now showstopper - underlying library doesn't…
c167074 
… support images yet!
@stoerr
resize images to <=512x512 when submitting
f9bb155
@stoerr
spec for content generation of classes for JSON serialization
e6fbc2d
@stoerr
code generation with developers toolbench
6af6c31
@stoerr
change toString to be more terse
1123f7e
@stoerr
remove openai/gpt3/java dependency because it isn't updated fast enough
1d69c42
@stoerr
rest of refactoring to remove the openai-gpt3-java dependency
de69c58
@stoerr
various fixes to make images work; extend config with visionModel
83c2351
@stoerr
image analysis works now for composum
8c3fa57
@stoerr
AEM creation - replace content creation selectors with servlet
7ba3757
@stoerr
fix path extraction
a20e981
@stoerr
show the image in the dialog and transmit inputImagePath
27ad042
@stoerr
implement inputImagePath into AICreateServlet
10a347e
@stoerr
bugfixes - AEM image as source works now.
244933c
@stoerr
Add describe image prompt
1a70314
@stoerr
add dependabot.yml
ee0acd2
@stoerr
minor fixes
056da9a
@stoerr
remove unused /content/experience-fragments/composum-ai folder
0fb849c
@stoerr
extend link collection to parent nodes to collect some siblings, too
90bb9d0
@stoerr
scroll to action bar on generation in creation dialog
06e626c
@stoerr
add some cases to html to markdown
a70f653
@stoerr
more url fixes
c826ac3
@stoerr
fixes related to image analysis
3388037
@stoerr
update documentation
ec21b71
@stoerr stoerr self-assigned this Jan 12, 2024
@stoerr stoerr added the enhancement New feature or request label Jan 12, 2024
@stoerr stoerr merged commit a51eb66 into develop Jan 12, 2024
2 checks passed
@stoerr stoerr deleted the feature/imagerecognition branch January 12, 2024 16:20
@stoerr stoerr mentioned this pull request Jan 12, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@stoerr stoerr

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@stoerr